Haptic interaction method, tool and system

ABSTRACT

The current invention concerns an improved method for communicating and processing motion output signals from agnostic haptic, wearable devices to the host system that runs, manages and modifies a type of an application. Specifically, the application of the inventions specifically is relevant to the domain of gaming, health, simulations, human-machine interfacing, robotics, control of unmanned vehicles/tools and the likes.

TECHNICAL FIELD

The invention pertains to the technical field of interaction of haptictools with applications being run on a host system via an improvedmethod for communicating and processing motion output signals fromagnostic haptic, wearable devices to the host system that runs, managesand modifies said application, preferably virtual realities andaugmented realities. Specifically, the application of the inventionsspecifically is relevant to the domain of gaming, health, simulations,human-machine interfacing, robotics, control of unmanned vehicles/toolsand the likes.

BACKGROUND

The objective of the invention is providing an improved and moreimmersive method of interaction of a user with an application,preferably with a virtual or augmented reality, via an improved handlingdevice and an improved method of processing the signals registered bythe handling device, to intuitively convey the actions from a user intoa desired effect in the application (potentially capable of creating aVR or AR), without excessive demands for setting up the handling devicewith respect to the host system, as is the case in many currently knownembodiments.

Known methods and specifically haptic handling devices for interactionwith electronic applications, specifically virtual (and augmented)reality, have been developed specifically to suit the needs of aspecific host system (for instance computer, smartphone, PlayStation,Xbox and others) or particular types of applications (shooter, flightsim, sports and others), and lack the versatility to be used fordifferent occasions, often even lacking the basic ability allowing theuse in other situations (often developers won't allow a joystick for afirst gaming system to be usable with a second different gaming system)and in fact prevents the interoperability of such tools. Not only doesit quickly become a very costly endeavor for a user to be equipped for awide variety of applications and/or systems, it also becomes moredifficult, as the particular of each of the devices (joysticks, steeringwheels, guns, etc.) will be intrinsically different in use, and won'tallow a user to easily become familiar with a wider range ofapplications.

A more generic range of such tools has also been manufactured, howeverthese typically require a separate, specific program (plug-in, SDK orothers) to be installed in the host system that needs a specific update,requires reprogramming or adaptations for each separate application typethat is to be run on the host system.

Both categories of handling devices allow objects to be moved,visualizations to be adapted, or the actuation of other parts of (VR orAR) environments by detecting motion of the user via sensors which sendsignals to a host system (computer, Xbox, PlayStation, etc.).

Furthermore, it is to be considered that the relevance of most prior arthandling devices with respect to the concept of this invention, isnonexistent, as prior art handling devices typically take a specificshape relevant to their application (as said, steering wheel, rifle, . .. ) and as such is only usable in a limited number of applications, manyof these even departing from the physical realities of the tools infact. For instance, a rifle tool for gaming will typically serve both aspistol, rifle, bazooka, etc. even without having the correct shape. Evenfurther, a joystick will typically be employed as a myriad of othertools, ranging from weapons to steering wheels, to even a hand andfingers. All of these handling devices will require a specific piece ofprogramming as mentioned in order to communicate efficiently with thehost system the application is running on to achieve the desired actionsin the application, and specifically in a virtual or augmented reality.The intent of this invention is to let the handling device actuate onthe user's hand in order to properly convey the experience of handlingthe correct tool applicable to each situation in an application,something which cannot be guaranteed with many of the currentlyavailable tools.

A user cannot be expected to purchase an entire arsenal ofapplication-specific handling devices (steering wheels, steeringjoysticks, pistol(s), rifle(s), other weapons, flight joysticks—oftenagain specific to certain vehicles), but it is still desirable to obtainan optimally realistic experience in the interaction with theapplication, for instance a virtual or augmented reality. This isoptimally achieved by the invention proposed in this document, whereinit is chosen not to provide an improved tool with sensors andelectronics to be handled which conveys the actuations exerted on saidimproved tool, but instead aims to provide a handling device whichconveys the actuations being exerted on a ‘dead’ object by the handlingdevice, typically said handling device being shaped to fit around a bodypart of the user, generally one or more hands (and/or fingers) and/orarms of the user. This furthermore achieves to simplify the way a usercan, with only few and easy actions, can provide control signals to ahost system with very high accuracy, this as opposed to less optimalhandling devices (for instance a joystick for a shooter application)that will require numerous and/or complex manipulations in order toprovide the desired signal to the host system.

A last important drawback of many of the known ‘interaction tools’ to beused along an electronic processing system such as PlayStation etc., isthat these frequently require additional infrastructure, such ascameras, sensors, wiring to accurately track the movements of the user.Not only is this a costly investments, it restricts the applicabilityand mobility of the interaction tools (needs enough free space, requiresthe relocation and possible installation of these infrastructuralelements when using the interaction tools in a new location, andgenerally makes the processing of the data more difficult as moreparameters are to be included in calculations).

There remains a need in the art for an improved handling device shapedto fit a body part of the user capable of detection actions andmanipulations of the user's body part, specifically the hand and fingersin most cases, and of transmitting these manipulations and actions assignals to the host system running the (VR or AR) application, whichhost system is adapted to automatically translate the signals from thehandling device of the invention into signals typical for the type ofhost system and the type of (VR or AR) application being run. Ingeneral, the invention aims to provide a simplified method ofinteraction between a handling device and a host system for running,managing and modifying (VR or AR) application environments.

The applicant takes note that some improvements were made in this field,for instance discussed in such documents as US 2016/054797 A1, US2016/279514 A1 and US 2016/162022 A1, though the solutions offeredtherein are found lacking. The invention disclosed in the presentapplication intends to further improve on the mentioned documents, aswell as anything else existing in the field.

The present invention aims to resolve at least some of the problemsmentioned above.

SUMMARY OF THE INVENTION

The present invention provides an improved method for agnosticinteraction with a host system running a type of application (preferablyagnostic interaction with a virtual reality (VR) or augmented reality(AR) produced by a host system, whereby said host system is suitable forrunning a type of VR and/or AR application), comprising the followingsteps:

-   -   a. detecting one or more hand manipulations of a user via one or        more haptic devices, preferably one or more haptic gloves, and        providing one or more motion output signals, associated to the        hand manipulations, to a processing unit on the one or more        haptic devices;    -   b. providing a system conversion library to said host system,        said system conversion library comprising a plurality of system        translation profiles, each system translation profile        distinctive to a specific type of host system, and each system        translation profile suitable for mapping a conversion of motion        output signals from the one or more haptic devices to one or        more associated control signals recognizable for said specific        type of host system, preferably whereby said system conversion        library comprises system translation profiles distinctive to at        least two or more of the following types of host system:        personal computer, two or more specific home video game        consoles, smartphone, robots, drones, smart tvs, smart glasses        and similar;    -   c. processing the provided motion output signals in the        processing unit on the one or more haptic devices;    -   d. transmitting the processed motion output signals to the host        system, preferably via a wireless communication standard, more        preferably via Bluetooth;    -   e. translating said motion output signals by the host system        into one or more associated control signals according to the        system translation profile distinctive to the host system        running the application; and    -   f. processing said associated control signals in the application        and modifying features therein according to at least said        control signals, by the host system. Preferably, the application        is a VR and/or AR application, and the step comprises modifying        features of the virtual reality or the augmented reality        according to the control signals.

In an alternative embodiment (again, preferably with the methodproviding agnostic interaction with a virtual reality (VR) or augmentedreality (AR) produced by a host system, whereby said host system issuitable for running a type of VR and/or AR application, and whereby thestep f. of processing the control signals preferably comprises modifyingfeatures of the virtual reality or the augmented reality according tothe control signals), the method comprises the following steps:

-   -   a. detecting one or more hand manipulations of a user via one or        more haptic devices, preferably one or more haptic gloves, and        providing one or more motion output signals, associated to the        hand manipulations, to a processing unit on the one or more        haptic devices;    -   b. providing a system conversion library to said processing        unit, said system conversion library comprising a plurality of        system translation profiles, each system translation profile        distinctive to a specific type of host system, and each system        translation profile suitable for mapping a conversion of motion        output signals from the one or more haptic devices to one or        more associated control signals recognizable for said specific        type of host system, preferably whereby said system conversion        library comprises system translation profiles distinctive to at        least two or more of the following types of host system:        personal computer, two or more specific home video game        consoles, smartphone, robots, drones, smart tvs, smart glasses        and similar;    -   c. processing the provided motion output signals in the        processing unit on the one or more haptic devices;    -   d. translating said motion output signals by the processing unit        into one or more associated control signals according to the        system translation profile distinctive to the host system        running the application;    -   e. transmitting the one or more associated control signals to        the host system, preferably via a wireless communication        standard, more preferably via Bluetooth; and    -   f. processing said associated control signals in the application        and modifying features therein according to at least said        control signals, by the host system

With the term “haptic device” as used herein, the document refers to awearable item, preferably suitable for being worn on the hand and/orfingers of a user such as a glove, provided with a number of sensors todetect a wide array of hand manipulations, as will be further discussed.Note the importance that the haptic device is not a handling device asperceived in prior art documents, wherein the handling device is a toolto be handled by a user (typically manually), but instead acts as theuser (or a body part thereof, typically a hand).

With the term “host system” as used herein, the document refers to amyriad of possible devices capable of running applications (programs,games, etc.), and preferably capable of producing a VR or AR environmentand of visualizing said environment (possibly via one or more displaydevices of some kind that can be built in the host system or can beplugged in—wirelessly and/or wired). This host system can be a generalcomputer, laptop, tablet, smartphone, gaming console (PlayStation, Xbox,Wii, and others), smart-tv, etc or dedicated AR and/or VR systems. Thesecan for instance be supplemented by VR/AR headsets or glasses. Note thata smartphone can be a host system, while the application that is run canfor instance be receiving a telephone call and managing said telephonecall, amongst other things such as running the visual display menu ofthe smartphone.

With the term “hand manipulations” as used herein, the document refersto a number of absolute and/or relative movements or actions by the handand/or parts thereof.

The term “type of application” as used herein, is meant a general areaor field to which the application belongs. Typically, for gaming thiscan be, as previously mentioned, the type of game such as shooters, RPG,sports (with several subtypes for different sports), racing (again withseveral possible subtypes), flying, etc. Note that many so-called typesof ‘simulator applications’ will also belong to the category of gaming.It is to be noted that in the convention handling devices, where thehandling device is the tool to be handled, the handling device is verydifferent depending on the application type, while the invention solvesthis problem expertly by in fact acting as the hand (or a shadow versionthereof) of the user (or another body part depending on the particularembodiment of the handling device of the invention).

The term “glove” or “haptic glove” can furthermore refer to anexoskeleton for a hand. The same applies to haptic devices of theinvention for other body part(s), these can similarly be provided as atype of exoskeleton for said body part(s).

In a second aspect, the present invention provides an electronicsystem-implemented method for preprocessing motion output signals in ahost system suitable for running a (virtual reality (VR) and/oraugmented reality (AR)) application, and suitable for creating aninteractive (virtual reality (VR) and/or augmented reality (AR))environment, received from one or more haptic devices by the hostsystem, preferably one or more haptic devices as described in document,comprising the step of translating said received motion output signalsby the host system into one or more associated control signals accordingto a predetermined system translation profile from a system conversionlibrary, said system conversion library comprising a plurality of systemtranslation profiles, each distinctive to a specific type of hostsystem, said predetermined system translation profile being distinctiveto the type of the host system running said application and preferablyalso to the type of application being run on the host system runningsaid application.

In a further aspect, the present invention provides an electronic systemfor creating, managing and interacting with a (virtual reality (VR)and/or augmented reality (AR)) application environment, comprising:

-   -   a. an electronic host system configured for producing the (VR        and/or AR) environment, said host system being suitable for        running a type of (VR and/or AR) application, and for modifying        the produced (VR and/or AR) environment according to received        control signals;    -   b. at least one at least partly flexible haptic devices suitable        to be worn on a user's hand, comprising:        -   a. a plurality of sensors configured for detecting hand            manipulations of the user's hand and transmitting associated            motion output signals, at least comprising: bending of one            or more fingers and pressure being exerted by one or more            surface on a surface;        -   b. a processing unit electronically connected to the            plurality of sensors, said processing unit being adapted for            receiving the associated motion output signals from one or            more of the plurality of sensors, configured for processing            the associated motion output signals, and adapted for            transmitting the processed motion output signals to the host            system via a wireless communication standard, preferably via            Bluetooth;            whereby said host system is provided with program            instructions to execute the electronic system-implemented            method as described in this document.

In a further aspect, the invention provides an electronic system forinteracting an electronic host system running a type of application, andfor modifying the application according to received control signals,said electronic system for interacting comprising at least one at leastpartly flexible haptic devices suitable to be worn on a user's hand,comprising:

-   -   i. a plurality of sensors configured for detecting hand        manipulations of the user's hand and transmitting associated        motion output signals, at least comprising: bending of one or        more fingers and pressure being exerted by one or more surface        on a surface;    -   ii. a processing unit electronically connected to the plurality        of sensors, said processing unit being adapted for receiving the        associated motion output signals from one or more of the        plurality of sensors, configured for processing the associated        motion output signals, said processing unit being configured to        recognize the type of host system, and adapted for transmitting        the processed motion output signals to the host system via a        wireless communication standard, preferably via Bluetooth;        whereby said processing unit is provided with a system        conversion library, said system conversion library comprising a        plurality of system translation profiles, each system        translation profile distinctive to a specific type of host        system, and each system translation profile suitable for mapping        a conversion of motion output signals from the one or more        haptic devices to one or more associated control signals        recognizable for said specific type of host system, preferably        whereby said system conversion library comprises system        translation profiles distinctive to at least two or more of the        following types of host system: personal computer, home video        game console, smartphone, dedicated VR and/or AR hardware,        robots, drones, smart tvs, smart glasses and similar. Said        system conversion library is typically stored in a memory        element of the processing unit, or on the haptic device and        accessible by said processing unit.

Preferably, the preceding method specifically provides an electronicsystem for interacting with a virtual reality (VR) and/or augmentedreality (AR) produced by an electronic host system, said host systembeing suitable for running a type of VR and/or AR application, and formodifying the produced VR and/or AR according to received controlsignals.

Alternatively, the invention provides an electronic system for running,managing and interacting with a type of application, preferably anapplication producing a virtual reality (VR) and/or augmented reality(AR), comprising:

-   -   a. an electronic host system configured for running the        application, and for modifying the application according to        received control signals;    -   b. at least one at least partly flexible haptic devices suitable        to be worn on a user's hand, comprising;        -   a. a plurality of sensors configured for detecting hand            manipulations of the user's hand and transmitting associated            motion output signals, at least comprising: bending of one            or more fingers and pressure being exerted by one or more            surface on a surface;        -   b. a processing unit electronically connected to the            plurality of sensors, said processing unit being adapted for            receiving the associated motion output signals from one or            more of the plurality of sensors, configured for processing            the associated motion output signals, and adapted for            transmitting the processed motion output signals to the host            system via a wireless communication standard, preferably via            Bluetooth;            whereby said host system is provided with a system            conversion library, said system conversion library            comprising a plurality of system translation profiles, each            system translation profile distinctive to a specific type of            host system, and each system translation profile suitable            for mapping a conversion of motion output signals from the            one or more haptic devices to one or more associated control            signals recognizable for said specific type of host system,            preferably whereby said system conversion library comprises            system translation profiles distinctive to at least two or            more of the following types of host system: personal            computer, home video game console, smartphone, dedicated VR            and/or AR hardware, robots, drones, smart tvs, smart glasses            and similar. Additionally, said system conversion library            can be supplemented for both aspects with an application            conversion library, or combined therewith.

Preferably, the electronic host system is configured for producing theVR and/or AR, said host system being suitable for running a type of VRand/or AR application, and for modifying the produced VR and/or ARaccording to received control signals.

DESCRIPTION OF FIGURES

FIG. 1 shows a schematic representation of a haptic device according tothe invention and an electronic system for running a type of interactiveapplication, and optionally creating and managing VR or AR environments.

DETAILED DESCRIPTION OF THE INVENTION

The present invention concerns an improved method and haptic device foragnostic interaction with a host system running a type of (interactive)application, preferably a VR and/or AR application.

Unless otherwise defined, all terms used in disclosing the invention,including technical and scientific terms, have the meaning as commonlyunderstood by one of ordinary skill in the art to which this inventionbelongs. By means of further guidance, term definitions are included tobetter appreciate the teaching of the present invention.

As used herein, the following terms have the following meanings:

“A”, “an”, and “the” as used herein refers to both singular and pluralreferents unless the context clearly dictates otherwise. By way ofexample, “a compartment” refers to one or more than one compartment.

“Comprise”, “comprising”, and “comprises” and “comprised of” as usedherein are synonymous with “include”, “including”, “includes” or“contain”, “containing”, “contains” and are inclusive or open-endedterms that specifies the presence of what follows e.g. component and donot exclude or preclude the presence of additional, non-recitedcomponents, features, element, members, steps, known in the art ordisclosed therein.

The term “application” generally refers to a program capable of beingrun on an electronic system. Typically, an application creates anenvironment in which the user can perform a number of tasks and/or issuea number of commands.

The term “agnostic interaction” refers to the interaction between hostsystem and haptic device being generally independent of host system,thus not requiring additional plug-ins or other software to bedownloaded in order to be able to communicate properly. The latter isthe case in most computer systems when being coupled with a notbrand-specific (complex) electronical device such as a haptic glove,which is one of the problems the invention wishes to address.

The term “nano-compressor” refers to a small-scale compressor suitablefor inflating/deflating air chambers with gas (air). The small scaleallow that a plurality can be mounted on a haptic glove without causingsignificant discomfort to a user, nor influence the actions of the user.

The recitation of numerical ranges by endpoints includes all numbers andfractions subsumed within that range, as well as the recited endpoints.

In a first aspect, the invention provides an improved method foragnostic interaction with a host system running a type of application,preferably an improved method for agnostic interaction with a virtualreality (VR) or augmented reality (AR) produced by a host system, saidhost system suitable for running a type of VR and/or AR application, asdescribed previously in both the first as in the alternative embodiment.

As mentioned earlier, prior art devices are often system-specific anddeny the user interoperability with other systems, or require separate,specific software for each handling device to run on each system. Theapplicant has overcome this by providing a more generic handling devicethat in the first place is meant to detect hand manipulations, insteadof handling device manipulations. This will allow for a far morerealistic representation of the action the user intends to convey in the(VR or AR) application, and can furthermore be far more intricate anddetailed than actions possible on a typical joystick (or even keyboard)or the likes. These more traditional controllers only have a limitedamount of variety in the way it can be manipulated, and thus a far morelimited amount of possible signals/actions that can be transmitted (soin order to allow more actions by a user in the (VR or AR) applicationenvironment, combinations of signals will become necessary, making theinteraction more complex and less natural). Furthermore, it allows theuser to handle the real or lifelike (tangible) objects (for instancegolf club, rifle, . . . ) to give a better indication of their actualweight, dimensions, and feel, to give a more realistic experience.

By using a wearable haptic device, in order to better capture the true,desired movements and manoeuvers of the user, this allows for a widerrange of motion output signals which can lead to issues in ‘translating’these to the host system in order to create the desired effect in the(VR or AR) application environment. In the few existing prior artsystems similar to the haptic device of the invention, this is usuallyremedied by limiting the application versatility of such wearable hapticdevices (in fact, creating them for a single purpose only), whereas theobject of this invention does not intend to be limited in its use incertain applications, but instead offers a universal applicability ofthe wearable haptic device. Prior art devices such as a controller (Wiior similar) have only a limited range of motions and nuances it candetect, although depending on the particular application that is run,there exists a wide range of possible motions and nuances that need tobe detected. For instance, a golf game will have the user hold andmanipulate a controller in a very particular way, while a helicopter simwill have the user manipulate the controller in an entirely differentfashion and will need to detect very different actions. By following anddetecting the exact motions, pressures, etc. of the user, the inventionat hand is able to correctly capture these nuances.

This is furthermore achieved by the method in a possible embodimentworking with an application conversion library comprising a number ofpredefined, application-specific ‘translation profiles’, or applicationtranslation profiles, which are meant to process the motion outputsignals registered by the haptic device, into the control signals thehost system is expecting to receive, or in other words, allows thehaptic device to emulate the devices natively supported by the hostsystem by translating the motion output signals of the haptic deviceaccordingly.

The most commonly used application types are limited (shooter, flightsim, racing, medical intervention, sports, etc.), although having amultitude of variants in all these categories (type of guns/rifle, typeof plane/chopper/ . . . , type of car/ . . . , and so on). As noted, bychoosing to providing a haptic device shaped like the body part (handtypically), there is no longer a problem with these different variants,as they will provide the same or similar signals to the host system. Thedifference in types of applications is then solved by providing a numberof predefined translation profiles adapted to specific applications toensure that the correct control signals are provided to the host systemto be used in the (VR or AR) application. Note that in case of veryspecific application types which were not provided, a translationprofile can be set up by the user easily, and saved for future use. Thisprofile can be saved locally on the haptic device and/or on the hostsystem.

The same applies to possible host systems, and the applicant has handilymade us of this by providing translation profiles for these hostsystems. In practice a number of popular home video game consoles are inexistence but are not compatible with each other, especially with eachother's controllers. The applicant conveniently solves this by providinga system conversion library with a number of specific translationprofiles for different host systems, system translation profiles. Mostconveniently, these can be set up for most known consoles, andadditionally other (gaming) processing systems (smartphones, computers,laptop, dedicated AR or VR systems, etc.). Again, specific systemtranslation profiles for ‘unknown’ host systems can be set up by theuser and saved on the haptic device and/or on the host system itself.

The applicant in a first embodiment of the method according to theinvention, provides a system conversion library on the host system(preferably as a downloadable executable, for instance can be downloadeddirectly from the haptic device) that comprises a multitude ofpredefined system translation profiles, designed to correctly‘translate’ or convert motion output signals from the haptic device intocontrol signals usually received by the host system to define thedesired action. Typically, there exist a number of popular andwidespread gaming consoles, as well as other host systems (smartphone,specially dedicated VR/AR processors, pc, laptop, . . . ), and for thesesystems, a predefined system translation profile can be set up, whilefor more uncommon systems, this can be defined by a user. Uponrecognition of the host system, the correct system translation profileis selected and used in the conversion. Alternatively, in a secondembodiment, the system conversion library is provided to the hapticdevice which executes the translation according to the principlediscussed above.

The transmitting of processed motion output signals from the (processingunit of the) haptic devices to the host system runs through wirelesscommunication, although wired communication is possible as well.Preferably, the signals are communicated via Bluetooth (Bluetooth LowEnergy—BLE), however, Wi-Fi or other wireless communication standardscan be used (alternative to or in addition to Bluetooth), such asZigBee, etc., as this allows for a fast, reliable connection withouthaving to deal with cumbersome wires during the interaction, which canhinder the immersive experience. Such a communication can be achieved bya transceiver microchip (or a component with similar function)implemented in the processing unit.

In this document, the shape of the haptic device is generally referredto as (one or more of) a wearable glove, however, it should in no way beseen as limited to that. It could for instance also take the shape of ahead mask, a head gear, glasses, footgear, body gear and combinationsthereof. For instance, a combination of a wearable glove with furtherarm gear (for instance worn on the biceps, or even in the elbow) couldbe of interest to specifically position to detect the relative positionof the hand to the arm, the angle with which the arm is bent, how it isgenerally aimed, etc. Furthermore, it is to be noted that the hapticdevice may generally be referred to as a glove, but that this in no wayrequires the device to be entirely shaped as a standard glove. It canalso be generally shaped to resemble and/or fit a hand-like structure,for instance flexible surface in contact with the hand and connectedthereto (by a ring for example). However, most preferably, the handlingdevice is configured to follow the motions of the user's hand.Consequently, the handling device is made, entirely or partially, offlexible material. In addition, the handling device comprises its ownbase portion which comes into contact with the palm of the hand, and aplurality of peripheral portions, intended to come into contact with thefingers of the hand.

It should be considered that the haptic device may possibly be providedwith multiple (spatially separated and/or detachable) subordinatedevices or slaves which all connect to the first, ‘central’ hapticdevice. These subordinate devices can comprise one or more sensors (asdiscussed) and transmit the motion output signals detected by thesensors on the subordinate devices to the central haptic device. Thesesubordinate devices may for instance be a second glove, finger pads,goggles, a helmet, a microphone, or other pieces of clothing or gearthat the user may wear. The configuration of the subordinate devices andthe central haptic device correspond to a cascading sensor system inwhich there is no direct communication to the host system via thesubordinate devices.

In a further improvement, parts of the haptic device can be adapted toallow interaction with capacitive screens, for instance to allowoperation of a smartphone, tablet, etc. If the haptic device is shapedlike a glove, such capacitive-enabling portions will typically be on thetips of one or more fingers.

Generally speaking, the haptic wearable device will comprise asupporting structure, preferably flexible, to allow the device to beworn practically on a body part. In this document, most concern will bedirected towards a haptic device to be worn on a hand, but the reasoningapplies to any body part and the concept of the invention should not belimited unduly to a hand only. The supporting structure, a glove, isprovided with a plurality of sensors. Said sensors can comprise a numberof bending sensors along to detect bending of a body part of the user.In this case, these bending sensors will extend longitudinally and runalong at least part of the fingers of a user to detect the bendingthereof. More generally speaking, the bending sensors will extend alongperipheral portions of the haptic device, extending away from a centralportion thereof. Nonetheless, bending sensors may as well be present inthe central portion of the haptic device.

Furthermore, a number of pressure sensors can be provided to detect thepressure on a portion of the haptic device against a surface. Typicallythese pressure sensors will at least be provided on peripheral ends ofthe haptic device (finger tips), and preferably additionally on a numberof other (intermediate) positions. For instance, an interesting point toprovide these is on the transitioning point from the metacarpals to thephalanges on the palm side of the hand.

A third kind of sensors are so-called haptic sensors, which typicallyextend longitudinally along peripheral portions of the haptic device,extending away from a central portion thereof, although the sensors maypreferably be present in the central portion of the haptic device aswell. These haptic sensors are designed to impart a feeling ofresistance, and optionally pressure (to indicate a user is touchingsomething). Typically, these sensors comprise inflatable elements tomake a user experience more or less resistance. Note that these sensorsare typically placed on the side of the supporting structure that isexpected to come into contact with an object or surface that is not theuser itself, while bending sensors are usually placed on the oppositeside.

Additionally, other types of sensors may be present, for instance agyroscope to determine the position and orientation of the haptic devicewith respect to a stationary reference. The gyroscope may to thatpurpose be an “attitude and heading reference system” (AHRS), typicallycomprising sensors on three axes to provide attitude information on thedevice (amongst which roll, pitch and yaw). The sensors on the axes cancomprise gyroscopes, accelerometers and magnetometers. Alternatively, aninertial measurement unit (IMU) can be employed as well, from which thedata is then processed in a different component or device. A barometermay furthermore be included.

Temperature sensors may be included in the haptic device as well,adapted to register temperature variations, in order to modify themeasurements by one or more of the other sensors (for instance from amagnetometer).

Also, so-called dead reckoning sensors may be present.

The haptic device may comprise temperature feedback actuators, which canvary their temperature depending on the feedback signals that are sentto the haptic device, and which actuators can convey the temperature tothe user. This is particularly useful for realistic simulationapplications (survival simulators, combat simulators, flight simulators,etc.) where temperature is a very real parameter that can influence thebehavior and performance of a user tremendously. Preferably, thetemperature feedback actuators can be set to a certain minimal andmaximal temperatures for protection, and can be set up to changetemperature to a lesser degree (for instance with respect to a basetemperature, for each degree the simulation temperature is increased orreduced, the feedback temperature is increased or reduced for a fractionof the increase, preferably up to a certain temperature limit bothupwards and downwards). It is to be noted that these actuators may alsobe used to impart the feeling of pain/induce pain on the user (again,with proper safety precautions) by exposure to high or low temperatures.

The haptic device may comprise vibration feedback actuators, which canproduce vibrations. The vibration feedback actuators are close to thesurfaces of the haptic device that touch the user when wearing orwielding the haptic device in order for the user to feel the vibrations.

The haptic device may comprise touch sensation feedback actuators thatare capable of inducing surface touch feeling on the user wearing thedevice. A possibility for the implementation can be through (functional)electric stimulation of the fingertip receptors (or receptors on otherparts of the body of course) of the user in order to emulate the actualelectrical signal sent to the peripheral nervous system when a surfaceof a material is perceived (smooth, rough, jagged, soft, hard,gelatinous, etc.). The electrical stimulation can for instance inducelow level muscle spasms, which creates a tangible/tactile sensation.Note that the surface does not necessarily need to be solid, but as saidcan be gelatinous, elastic, etc., resulting in a varying resistance thatwould normally felt by a user. This varying resistance can be properlyemulated by modulating the intensity of the electrical stimulation asthe user's hand/finger (or other body part) moves. Depending on the typeof feedback that is desired, the electrical stimulation can vary inintensity and duration. The electrical stimulation may be functionalelectrical stimulation (FES), transcutaneous electrical nervestimulation (TENS) and/or neuromuscular electrical stimulation (NMES).In a possible addition, temperature feedback actuators may also bepresent which use electric signals to impart the feeling of heat, cold,and generally temperature changes, to the user. This further strengthensthe authentic feeling of the surface contact to the user.

The haptic device may comprise pain feedback actuators that are capableof inducing pain on the user wearing the device. A possibility for theimplementation can be through electric stimulation of the pain receptorsof the user (electroshock), or through heat. These possibilities allowthe application to impart a more realistic feeling in certainapplications (for instance in combat simulation).

It is to be noted that not all types of sensors are necessarily presenton the haptic device of the invention. It is possible to only havebending sensors, or only pressure sensors or only haptic sensors, or thecombination of pressure and bending sensors, or pressure and hapticsensors, or bending and haptic sensors, or all three, possiblesupplemented with one or more of the other options discussed.

As mentioned before, the haptic device comprises a processing unit, witha CPU or CPU capability. Said processing unit is operably connected tothe mentioned plurality of sensors in order to obtain the measurementsand motion output signals from the sensors. Preferably, this connectionis wired (electrical wiring), although a wireless connection could bepossible, especially should there be made use of a number of hapticdevices, which are in communication with a single central processingunit. The processing unit is configured to process the obtain motionoutput signals from the sensors as is necessary (can be dependent on thehost system requirements), and preferably applies an error estimationand correction to this data from the sensors. Further processing is alsopossible, for instance, compression and/or encryption before forwardingthe data to the host system, where it is provided to a specific programor piece of software that is configured to process the received motionoutput signals further into the control signals native to the hostsystem. Once this translation step is completed, the control signals areprovided to the host system to be processed in the (VR or AR)application that is running, thereby typically modifying features of theproduced (VR or AR) environment, or changing it in some way or another.Optionally, feedback signals can then be provided back to the hapticdevice, and thereby, to the user.

As mentioned before, different host systems may have different‘expected’ control signals to convey one or more particular actions tothe host system. Therefore, the applicant applies in its proposed methoda system translation profile from a system conversion library, whichtranslation profile is specific to the type of host system on which the(VR or AR) application is run. A wide variety of profiles for ‘standard’systems can be pre-programmed, thus easily allowing a plug-and-playoperability of the method proposed by the applicant, whereby the correcttranslation profile is chosen for the host system. In other words, ifthe haptic device is ‘loaded’ (recognized by the host system, acceptedby a user) by a host system, the specific program of the method willrecognize the type of host system, for instance a PlayStation, and willconvert the received motion output signals from the haptic device'sprocessing unit into the correct control signals a PlayStation hostsystem would expect.

It is to be understood furthermore that the processing unit adapts themotion output to a suitable (interface) standard for communication withthe host system. This may be Bluetooth, Zigbee, WiFi, wired (USB) orother standards.

In a possible embodiment, the processing unit further comprises one ormore memory element which (temporarily) store recent motion outputsignals (or the associated control signals if the motion output signalshave been processed) in case there is no connection to the host system.Once a connection has been established later on, the stored signals canthen be communicated to the host system, which can then translate thereceived signals into actions or movements. For instance, the hapticdevice may store movement information (acceleration, tilt, etc.)temporarily. Once the connection is (re)established, the host system canthen recreate the path/movement that was acted out while no connectionwas present. This recreation can be accomplished by using dead reckoningprocesses. The memory element stores information on relative discretemovements (over discrete portions of time), which can then be used toreconstruct the entire movement by combining them.

In an improvement to the first embodiment of the method according to thefirst aspect of the invention, the method furthermore comprises a stepof providing an application conversion library to said host system, saidapplication conversion library comprising a plurality of applicationtranslation profiles, each application translation profile distinctiveto a specific type of application, and each application translationprofile being adapted for mapping a conversion of received motion outputsignals from the one or more haptic devices to one or more associatedcontrol signals suitable for said specific type of application,preferably whereby said application conversion library comprisesapplication translation profiles distinctive to at least two or more ofthe following types of application: shooter application, flightsimulator application, driving or racing application, remote pilotingapplication, smart device interaction; whereby the step of translatingsaid motion output signals into one or more associated control signalsaccording to the system translation profile distinctive to the hostsystem running the application, is executed furthermore according to theapplication translation profile distinctive to the type of theapplication being run on the host system.

In an improvement to the second (alternative) embodiment of the methodaccording to the first aspect of the invention, the method furthermorethe step of providing an application conversion library to saidprocessing unit, said application conversion library comprising aplurality of application translation profiles, each applicationtranslation profile distinctive to a specific type of application, andeach application translation profile being configured for mapping aconversion of received motion output signals from the one or more hapticdevices to one or more associated control signals suitable for saidspecific type of application, preferably whereby said applicationconversion library comprises application translation profilesdistinctive to at least two or more of the following types ofapplication: shooter application, flight simulator application, drivingor racing application, remote piloting application, smart deviceinteraction; whereby the step of translating said motion output signalsinto one or more associated control signals according to the systemtranslation profile distinctive to the host system running theapplication, is executed furthermore according to the applicationtranslation profile distinctive to the type of the application being runon the host system.

In a furthermore preferred embodiment, the method comprises the step ofthe user manually determining the application translation profile, fromthe application conversion library, to be used for translating themotion output signals, furthermore allowing the user to create a newapplication translation profile for the type of application running onthe host system, whereby said new application translation profile ispreferably stored on the processing unit of the haptic device and/or onthe host system.

In a preferred embodiment of the first aspect, the method comprises astep of automatically determining the type of the host system by theprocessing unit. By automatically determining the type of the hostsystem (for instance from a range of common and/or more specific typesof host systems such as PlayStation, Wii, Xbox, PC, smartphone—againpossibly with different subtypes—, smart-tv's, etc.), the methodprovides an enhanced plug-and-play ability to the user, and moregenerally the haptic device the user is employing. Thus it is no longerrequired that the user performs an elaborate set-up, or has to selectthe ‘correct’ host system (more important even as not all types will beas obvious as the ones listed above, especially when dealing with highlyspecialized host systems, for instance in medical applications,professional flight sims, etc.). Where more and more manufacturers insmartphone, game console and similar electronic media industries, havestarted developing single-system controllers, joysticks, haptic devices,etc., this has the dual effect of excluding users from certainsystems/applications due to the exorbitant cost of these devices, butsecondly also requires the user to develop a ‘feel’ and adequacy foreach new device separately, instead of becoming extremely proficientwith a single tool, as is the intention of the invention at hand.

In a preferred embodiment of the first aspect, the method comprises astep of automatically determining the type of the host system by theprocessing unit, and in case of inability to determine the type of thehost system, allowing the user to either manually determine the type ofthe host system and/or prompting the user to create a new systemtranslation profile for the undetermined host system, whereby said newsystem translation profile is preferably stored on the processing unitof the haptic device and/or on the host system. Alternatively oradditionally, the new system conversion library is stored locally on thehost system. Similarly, a new application translation profile for theapplication conversion library can be set up by the user for unknowntypes of applications.

Again, by automatically determining the type of the host system (forinstance from a range of common and/or more specific types of hostsystems such as PlayStation, Wii, Xbox, PC, smartphone—again possiblywith different subtypes—, smart-tv's, etc.), the method provides anenhanced plug-and-play ability to the user, and more generally thehaptic device the user is employing. Thus it is no longer required thatthe user performs an elaborate set-up, or has to select the ‘correct’host system (more important even as not all types will be as obvious asthe ones listed above, especially when dealing with more specializedhost systems, for instance in medical applications, more professionalflight sims, etc.).

The step of automatically determining the type of host system may beimplemented through a number of methods, of which a few are elaboratedon in what follows:

A first method comprises a so-called ‘handshake’ operation in thecommunication protocol between the host system and the processing unit.The processing unit will request an identification of the host system (asystem code of any type to identify the host system), which theprocessing unit then compares to known identifications in its memory.This identification can for instance be obtained by the processing unitconsulting the library of the host system (any host system in the fieldcomprises such a library) and by analyzing (some of) the data in thelibrary, the type of host system may then be determined. It is to benoted that such a procedure would mean that no actions or modificationswhatsoever are required to integrate any existing host system in themethod of the invention. Alternatively, the host system may be modifiedto automatically provide an identification upon request of theprocessing unit however. Typically, the processing unit will have a listof identifications for most of the commonly used systems. Given thatthere are a limited number of ‘players’ on the market (for consoles,such a list could for instance comprise Sony, Microsoft, Nintendo, aswell as some smaller players, which list can then be further subdividedfor specific host systems of the players; for PC and laptops,smartphones and tablets, and other types of host systems, similar listsmay be readily made; of course new manufacturers that enter the fieldand new technologies can easily be added to this list) this list willnot be overly large, allowing an easy identification. It is to be notedthat such a list can easily be updated should new systems or even newproviders enter the market, for instance via a patch that isautomatically or manually downloaded onto the processing unit when it isconnected to the internet.

A second method is the integration of software and/or hardware into thehost system, which could communicate directly with the processing unit.Said piece of software and/or hardware is specifically adapted to thehost system, and can thus share the specifics with the processing unit,allowing an easy identification of the host system.

A third method would be by “trial and error”. The processing unitsbegins a synchronization process in which ‘random’ command signals arecommunicated to the host system. Preferably, the command signals thatare communicated are known to elicit a response from host systems of oneor more of the ‘known’ types. Based on the responses, the type of thehost system is then determined. This could be determined after a singleexchange, or after a few iterations, depending on how fast the hostsystem type is determined. Preferably, the command signals are sent inan order that matches the prevalence of host systems, as this wouldstatistically minimize the recognition time for determining the type.Alternatively, the command signals are sent in an order that matches thelast recognized host systems. Building further on this approach, thecommand signals can be sent in an order based on the frequency withwhich each type of host system has been recognized for the specificprocessing unit, or even a mix of two or more of these approaches (forexample, based on frequency of use and on last used host system). It isto be noted that after the determination of the type of host system, aconfirmation signal can still be sent to the host system to make surethat the determination was correct.

In a preferred embodiment, the haptic device thus is a haptic glovesuitable for being worn on a hand of the user, comprising sensors atleast adapted for detecting the following hand manipulations, andpreferably the intensity and/or speed and/or acceleration of saidfollowing hand manipulations: bending one or more fingers of the hand,relatively displacing and/or rotating one or more fingers of the handwith respect to one or more other fingers of the hand, pressure beingexerted by a portion of one or more fingers on a surface, pressure beingexerted by a portion of the hand on a surface, displacement of the handwith respect to the host system, rotation of the hand with respect tothe host system; and wherein said following hand manipulations, andpreferably said intensity and/or speed, is provided to the processingunit as associated motion output signals. A list of possible sensorssuitable for detecting said movements has previously been discussed.

Note that when referring to a body part such as ‘hand’ or ‘finger’, itis to be understood the device in fact detects the manipulation exertedon the part of the device pertaining to said body part.

In a preferred embodiment, the processing unit is, or is comprised in, asmartphone or electronic device, preferably suitable for operableinsertion in a virtual reality headset. More preferably, said smartphoneor electronic device comprising one or more sensors for determiningspatial displacement of the user: gyroscope, dead reckoning sensor,magnetometer and/or accelerometer; said method comprising the steps of:processing displacement signals from said one or more sensors fordetermining spatial displacement; providing additional and/oralternative signals to the host system based on said displacementsignals to the host system; and furthermore modifying features of theapplication (environment), preferably by modifying a virtual reality oran augmented reality by the host system according to at least additionaland/or alternative displacement signals.

Given the omnipresence of smartphones in today's world, the proposedmethod can specifically be adapted thereto. Additionally, the method canmake use of a number of sensors already present in the smartphoneitself, and thus not only rely on the sensors of the haptic device. Thiscan for instance be used as a correction on certain signals from thehaptic device, or can convey additional data (for instance, orientationof the head of the user—direction of line of sight of the user—when usedwith a headset). However, the sensors of the smartphone can especiallybe used with respect to detecting spatial displacement of the user.

In a preferred embodiment, the haptic glove is adapted for determiningthe intensity of said following hand manipulations, comprising a step ofcalibrating at least the motion output signals associated to theintensity of the bending of one or more fingers of the hand and to theintensity of pressure being exerted by a portion of one or more fingersof the hand. By having the user calibrate the intensity settings for themeasurements of the haptic device, he can further personalize the hapticdevice to his own needs, or those of the particular application, forinstance by setting activation thresholds. This option of calibrationcan be prompted at set-up for instance as a yes-or-no question. Ifchosen, the user can then give certain intensities of bending, pushing,pulling and/or more actions more specific meanings to be used in theapplication.

In a preferred embodiment, said one or more haptic devices comprise aplurality of reversibly inflatable air chambers on the haptic device forexerting pressure and/or force on a body part of the user, and aplurality of nano-compressors for inflating and deflating said airchambers, said method comprising a step of transmitting one or morefeedback signals to the processing unit of the one or more hapticdevices based on said modified features, whereby said feedback signalsare actuated by inflation and/or deflation of one or more of theplurality of the air chambers by one or more of the nano-compressors.

In a preferred embodiment, the associated control signals are adapted toreplicate expected input signals for the host system.

In a preferred embodiment, the step of processing the provided motionoutput signals in the processing unit on the one or more haptic devices,comprises applying a Kalman filtering algorithm on the motion outputsignals, preferably a linear Kalman filtering algorithm. However, other,more general, types of filters than Kalman filtering can also be used.

By applying a filter, it is possible to reduce the level of noise andguarantee better, more reliable results. The Kalman algorithm iscomposed by two typical phases: the first one is a prediction of thesystem state, while the second step consists in updating the predictionmade according to the last measurement received. The algorithm runsrecursively in real time keeping track only of the previous state andthe last measurement. There are several versions of the Kalman filter,each one of them can yield better result according to its specificutilization, where the most common ones are:

-   -   Linear Kalman: Normalized sensor data and reference vectors are        fed into the Kalman filter, which uses statistical techniques to        optimally combine the data into a final orientation reading.        Provides the highest-accuracy orientation at the lowest        performance;    -   Alternating Kalman Filter: Uses the same Kalman filter as        before, but skips every other update step. Slightly less        accurate than the Kalman filter, but faster.    -   Complementary Filter: Fuses low-pass filtered        accelerometer/compass data with high-pass filtered gyroscope        data to provide an orientation estimate. Less accurate than any        Kalman filtering techniques, but provides significantly higher        performance.    -   Quaternion Gradient Descent Filter: Utilizes gradient descent        techniques to avoid the high computational overhead of        Kalman-based filters. Provides high performance and high        accuracy.

Preferably, the linear Kalman filter is chosen, and applied directly onthe processing unit of the haptic device (as well as if a differentfiltering algorithm would be used). Said filtering algorithm thencomputes the pitch, roll and yaw angle of the haptic device (and thustypically of the user's hand). In order to perform this computation, itobtains the raw data from a number of sensors, typically these sensorscomprise at least one or more of the following: accelerometer,gyroscope, magnetometer, barometer, which can be embedded on theprocessing unit. Upon receipt, a calibration or normalization is usuallyperformed. Preferably, all four of the aforementioned sensors(accelerometer, gyroscope, magnetometer, barometer) are used for thisstep. The following step consists in the normalization and conversion ofthe raw data. They are converted in the desired measurement unit fromthe rough data given by the sensors, moreover some adjustments areperformed due to hardware characteristics. Once the preprocessing phasehas been made, the processing unit continues computing the filtered datathrough the application of the Kalman filter. As said, in order tocompute the pitch and roll estimation the algorithm takes advantage ofthe gyroscope's data and the accelerometer's ones, while, for the yawangle, the Kalman uses the gyroscope's measurements and the magnetometerones.

Typically, part of the method of the invention is being run on twoprograms or levels, a first, called the Core Process, and a second,called the Suite Process. The Core Process is the lowest level of thesoftware layer. It is the core of the integration between the hapticdevice and the existing host system. Traditional systems, in fact, arenot designed to work with sensors, activation thresholds or otherfeatures of the haptic device. It is therefore necessary to have a newcomponent that can map the signals it receives from the haptic deviceinto instructions that are understandable to the host system. Havingevery system its own peculiarities, it must be adapted to the specifichost system on which the method and haptic device must operate. Thismeans that once the Core Process has been created for a specificplatform, every application running on that platform or system can takeadvantage and utilize the provided technology.

The Core Process therefore, interacts with the host system. In order todo that, it must emulate the controller devices natively supported, suchcan be a mouse or a keyboard. Hence, if the user chooses, through theSuite Process, to assign to some movement the same function of the leftclick of the mouse, or a stroke on the keyboard, the Core Process willsend a left click of the mouse or that particular stroke of the keyboardto the host system. As for the mouse or the keyboard, the preprocessingmethod can be designed to emulate all the common controllers ondifferent kinds of host systems. New systems, however, can be designedto support the haptic device natively without requiring emulation.

As it has been said, the Core Process is system dependent. This meansthat, according to what kind of technological device the user wants tocontrol and what kind of system is running on it, a specificimplementation must be adopted. Possible devices are PCs, Smartphones,PlayStations, Xbox, or newer ones such as Drones or Apple TV. Thegeneral principle the proposed method takes advantage of, is that thesedevices already “react” to some inputs for which they have beendesigned; inputs such as mouse movements, keyboard strokes, radiosignals and many more. The method of preprocessing doesn't substitutethese inputs but rather replicates them when needed. To this purpose,the Core Process must be able to listen on a specified interface towhich the controller sends its inputs. The interfaces adopted for themethod are Serial in case of a wired solution or Bluetooth otherwise,but others can be adopted for specific usages. The Suite Process setsthe values of several variables in the code that determines how the CoreProcess executes its tasks. The commands that Core Process executes aregenerally system commands furnished by some native APIs. In the case ofa Windows system for example, the method can use C #functions to send akeyboard stroke or a mouse event to the running application.

The proposed method and underlying technology offers a wide range ofcustomizations that the user can set through an interactive procedure.The Suite Process will let the user specify his preferences using agraphical interface and gives the user the opportunity to test themclicking on a Test button. The first step of the guided procedure is toload a haptic device which the Suite Process will interact with. To dothat the user scans for available devices and chooses how it should beloaded. Possible options are to load a controller as left/right hand orarm. One or more controllers can be used at the same time. After havingloaded a device the user has to choose the settings he wants to apply toit. The method offers some predefined configurations profiles fordifferent applications that might be used “as is” or modified to betterfit one's own needs. Also, new profiles can be created from scratch andsaved for later usage. All the saved profiles are shown in a box wherethey can be selected with a simple click or a touch. Using a savedprofiles becomes particularly useful with the invention because of itshigh number of parameters combinations. The invention has in fact, ahigh level of granularity that can fit the needs of even the mostdemanding experience. Every application, moreover, can have its specificsettings and the user can decide to have a different profile for anyparticular usage. For instance, in the videogames field, when emulatinga helicopter piloting, the user will hold the cloche in a different waythan he would handle a golf club when playing golf. This means, asdescribed later, that the fingers at rest, or the range of theirmovements, will vary according to the context. A further key aspect isthat different applications may use different input controllers, such asa mouse or a joystick. The method and system can emulate thesetraditional controllers and lets the user choose which one has to beemulated.

In a third aspect, the invention provides an electronic system forrunning, managing and interacting with a type of application, preferablya type of application capable of producing a virtual reality (VR) and/oraugmented reality (AR), comprising:

-   -   a. An electronic host system for running the application, and        for modifying the application (environment) according to        received control signals; preferably however, it specifically        comprises an electronic host system configured for producing the        VR and/or AR, said host system being suitable for running a type        of VR and/or AR application, and for modifying the produced VR        and/or AR according to received control signals;    -   b. at least one at least partly flexible haptic devices suitable        to be worn on a user's hand, comprising:        -   a. a plurality of sensors configured for detecting hand            manipulations of the user's hand and transmitting associated            motion output signals, at least comprising: bending of one            or more fingers and pressure being exerted by one or more            surface on a surface;        -   b. a processing unit electronically connected to the            plurality of sensors, said processing unit being adapted for            receiving the associated motion output signals from one or            more of the plurality of sensors, configured for processing            the associated motion output signals, and adapted for            transmitting the processed motion output signals to the host            system via a wireless communication standard, preferably via            Bluetooth;            whereby said host system is provided with program            instructions to execute the electronic system-implemented            method as described in this document.

In a further aspect, the invention provides an electronic system forinteracting with an electronic host system running a type ofapplication, and for modifying the application (environment) accordingto received control signals. Preferably it provides an electronic systemfor interacting with a virtual reality (VR) and/or augmented reality(AR) produced by an electronic host system, said host system beingsuitable for running a type of VR and/or AR application, and formodifying the produced VR and/or AR according to received controlsignals.

Said electronic system for interacting comprises at least one at leastpartly flexible haptic devices suitable to be worn on a user's hand,comprising:

-   -   i. a plurality of sensors configured for detecting hand        manipulations of the user's hand and transmitting associated        motion output signals, at least comprising: bending of one or        more fingers and pressure being exerted by one or more surface        on a surface;    -   ii. a processing unit electronically connected to the plurality        of sensors, said processing unit being adapted for receiving the        associated motion output signals from one or more of the        plurality of sensors, configured for processing the associated        motion output signals, said processing unit being configured to        recognize the type of host system, and adapted for transmitting        the processed motion output signals to the host system via a        wireless communication standard, preferably via Bluetooth;        whereby said processing unit is provided with a system        conversion library, said system conversion library comprising a        plurality of system translation profiles, each system        translation profile distinctive to a specific type of host        system, and each system translation profile suitable for mapping        a conversion of motion output signals from the one or more        haptic devices to one or more associated control signals        recognizable for said specific type of host system, preferably        whereby said system conversion library comprises system        translation profiles distinctive to at least two or more of the        following types of host system: personal computer, home video        game console, smartphone, dedicated VR and/or AR hardware,        robots, drones, smart tvs, smart glasses and similar.

Said system conversion library is typically stored in a memory elementof the processing unit, or on the haptic device and accessible by saidprocessing unit.

Furthermore it is to be noted that the electronic device is preferablyadapted in order to accommodate the method of the invention for agnosticinteraction with an application (environment), preferably a virtualreality (VR) or augmented reality (AR). For one, preferably, the hapticdevice is configured to recognize or determine the type of the hostsystem, thereby allowing the correct selection of a proper systemtranslation profile for further communication with the host system bythe haptic device. Again, preferably the haptic device is furthermoreprovided with an application conversion library comprising a pluralityof application translation profiles for converting motion output signalsinto control signals acceptable by the host system. Preferably, thesystem and application conversion libraries are combined into a singleconversion library, comprising a plurality of system and applicationtranslation profiles. In addition, new system translation profiles forthe system conversion library can be created, for instance when facedwith an ‘unknown’ host system. The same applies for new applicationtranslation profiles (as these are often customized by users).

Alternatively, the invention provides an electronic system for running,managing and interacting with a type of application (environment),preferably running, managing and interacting with a virtual reality (VR)and/or augmented reality (AR), comprising:

-   -   a. an electronic host system for running the application, and        for modifying the application (environment) according to        received control signals; preferably this is an electronic host        system configured for producing the VR and/or AR, said host        system being suitable for running a type of VR and/or AR        application, and for modifying the produced VR and/or AR        according to received control signals;    -   b. at least one at least partly flexible haptic devices suitable        to be worn on a user's hand, comprising;        -   a. a plurality of sensors configured for detecting hand            manipulations of the user's hand and transmitting associated            motion output signals, at least comprising: bending of one            or more fingers and pressure being exerted by one or more            surface on a surface;        -   b. a processing unit electronically connected to the            plurality of sensors, said processing unit being adapted for            receiving the associated motion output signals from one or            more of the plurality of sensors, configured for processing            the associated motion output signals, and adapted for            transmitting the processed motion output signals to the host            system via a wireless communication standard, preferably via            Bluetooth;            whereby said host system is provided with a system            conversion library, said system conversion library            comprising a plurality of system translation profiles, each            system translation profile distinctive to a specific type of            host system, and each system translation profile suitable            for mapping a conversion of motion output signals from the            one or more haptic devices to one or more associated control            signals recognizable for said specific type of host system,            preferably whereby said system conversion library comprises            system translation profiles distinctive to at least two or            more of the following types of host system: personal            computer, home video game console, smartphone, dedicated VR            and/or AR hardware, robots, drones, smart tvs, smart glasses            and similar. Preferably, said system conversion library is            provided to the host system as a downloadable executable.            Note that the necessary executable or program can            furthermore be provided via a wireless connection from the            haptic device to the host system when the haptic device is            ‘loaded’ onto the host system.

Furthermore it is to be noted that the electronic device is preferablyadapted in order to accommodate the method for agnostic interaction witha virtual reality (VR) or augmented reality (AR) of the invention.Preferably the host system is furthermore provided with an applicationconversion library comprising a plurality of application translationprofiles for converting motion output signals into control signalsacceptable by the host system. Preferably, the system and applicationconversion libraries are combined into a single conversion library,comprising a plurality of system and application translation profiles.In addition, new system translation profiles for the system conversionlibrary can be created, for instance when faced with an ‘unknown’ hostsystem. The same applies for new application translation profiles (asthese are often customized by users).

Note that in both of the aforementioned embodiments (the further aspectand the alternative thereto), the preferred sensors and their preferredpositioning have been discussed already in this document, to which isreferred, along with advantages. Preferably, the haptic device comprisesa plurality of reversibly inflatable air chambers on the haptic devicefor exerting pressure and/or force on a body part of the user, and aplurality of nano-compressors for inflating and deflating said airchambers, said method comprising a step of transmitting one or morefeedback signals to the processing unit of the one or more hapticdevices based on said modified features, whereby said feedback signalsare actuated by inflation and/or deflation of one or more of theplurality of the air chambers by one or more of the nano-compressors.

The invention is further described by the following non-limitingexamples which further illustrate the invention, and are not intendedto, nor should they be interpreted to, limit the scope of the invention.

The present invention will be now described in more details, referringto examples that are not limitative.

EXAMPLES Example 1: Profile Configuration (System/Application ConversionLibrary)

In order to configure a device that has been loaded, the user can takeit through a guided procedure that will let him specify all the “zeros”.The user can specify what will be his starting position, how he willkeep his hand or fingers when at rest and what the maximum span of hismovements is. This phase allows the method to calibrate the values ofthe sensors and interpret them in a proper manner.

The method can for instance allow to set up ‘mouse settings’. Thisallows the user to adopt one of two possible positioning option. He canchoose between relative and absolute positioning, and set thesensitivity of her movements through an adjustable bar on the screen. Italso allows to specify how the mouse cursor must move when moving theuser's hand on a specific axis. Similarly, the user can specify hispreferences for the joystick emulation. Also in this case, he can choosethe commands to execute upon his hand's movements.

With the proposed haptic device and method, it is possible to configureall the sensors on the device's appendages (the glove's finger), forinstance via a menu option “Finger settings”. A window will appear wherethe user can select his options for the bending and pushing sensors. Thefirst type of sensors can be set to act in bending mode or stretchingmode. In bending mode, the user normally keeps his finger stretched andthe corresponding sensor is activated when the finger is bent. On thecontrary, in stretching mode, the user's finger is normally closed andits sensor is activated when the user opens it. Different kinds ofcommands can be triggered when these actions are performed. Forinstance, the user can decide to strike the ‘w’ character or to applythe left-click of the mouse when he bends his index finger. Moreover, itis also possible to set a command's specific activation threshold andassign, at the same sensor, two commands triggered at differentthresholds.

Similarly, it is possible to configure the pushing sensors in two modes:pushing and release mode. As in the precedent case, every sensor isactivated when the designated threshold is hit and the correspondingaction can be executed either when the sensor is pushed (push mode) oras soon as it is released (release mode). In pushing mode, it ispossible to configure two commands on the same sensor, according to theintensity which the sensor has been pushed with.

A further feature “Input Axis Settings” can let the user assign commandswhen moving one's own hand along some axis. It is possible, in fact, toassign a combination of key or mouse clicks to the user's hand motion.

Further options are “Acceleration Settings” and “Barometer settings”.The first allows to define what actions must be taken when the usermoves his hand either upwards/downwards or leftwards/rightwards with acertain acceleration. Also in this case there is an adjustable barthanks to which the threshold needed to trigger the command can be set.

“Barometer Settings” can be used to set a series of commands to betriggered at various heights of the user's hand. If the user issimulating a gun fight, for instance, he might want to activate the“sniper” function, when he keeps her hand at eye level, and use a simplegun otherwise. Thanks to the barometer, the method can perceive theheight of the user's hand and act accordingly.

Example 2: Haptic Glove

In a possible embodiment, the haptic device comprises a haptic glove(although two gloves may as well be possible) with a supporting basestructure (4) of a flexible, wearable glove, which is adapted to contactthe palm and back of the hand, and a number of peripheral portions (5)adapted to contact the fingers of the hand. Said haptic glove comprisesa number of sensors. Specifically, bending sensors (9) are provided toextend along the fingers, preferably on the backside of the finger (sameside as the back of the hand as depicted in FIG. 1), and are configuredand positioned to detect the bending of the peripheral portions (as aconsequence of the fingers or other body parts bending). The bendingsensors (9) can extend somewhat across the metacarpal region of the handin order to detect a more general bending of the fingers with respect tothe hand. Furthermore, a number of pressure sensors (7) are provided ata number of positions. In this case, said pressure sensors (7) are atleast provided on the ends of the peripheral portions, this time on thefront side (palm side) of the hand. Note that further pressure sensorsmay be provided at other points of the fingers of the gloves, and/oralso on the palm of the hand. Additionally (or alternatively), a numberof motion sensors (6) or haptic sensors, may be provided on the hapticglove as well. Typically these are at least provided on the ends of thefingers or peripheral portions again, although they can furthermore bepresent on the hand portion of the glove as well.

These motion sensors (6) can be supplemented by a platform for one ormore sensors, for instance comprising a gyroscope (11) which is in thiscase present at the base structure (4). Said gyroscope can (furthermore)comprise an AHRS, and/or a height sensor to detect the height of thehaptic device with respect to the ground. Furthermore, said platform maycomprise a temperature sensor and/or other mentioned sensors.

A processing unit (14) is provided on the haptic device (glove),preferably on the base structure (4), and is provided with a battery(15). Said battery can be rechargeable (mini-USB for instance) and/orreplaceable. The processing unit (14) is connected to, or at least ableto connect to, the sensors on the haptic device in order to obtain thedetected signals (8, 10, 12), generally called motion output signals (8,10, 12). Preferably this connection is through electric wiring, althoughwireless connection may also be possible, or a mix of both, for instancein case of several haptic devices (gloves, etc.) and only a singleprocessing unit for all of these.

Said processing unit (14) is configured to communicate with the hostsystem (100), preferably via a wireless communication standard (16).

Example 3: Smartphone Handling

In a possible embodiment, the haptic device, for instance in the form ofa haptic glove, can be used in combination with a smartphone. Once theconnection is established, the user can provide commands to thesmartphone via manipulations of the haptic device (as mentioned before,spatial—relative and/or absolute—movements such as translation,rotation, or via pushing, pulling, pressure, etc.). A system translationprofile for the smartphone can be already provided or can be created (orjust adapted) to allow the user to control the smartphone withoutnecessarily touching the smartphone. This could potentially allow a userto send a text message, making calls, changing settings, sendinge-mails, performing an online search, playing a video, playing music,etc. without handling the smartphone, and in fact without necessarilyeven seeing it. As an example, the user can via a first gesture generatea motion output signal which is translated by the system translationprofile for the smartphone into a control signal for the smartphone thatconstitutes that the user wishes to send a text message. The user canthen, via a second gesture, communicate that he wishes to create virtualkeyboard upon which he can then type the message and/or the addressee. Athird gesture could then send the text message.

Another example could allow the user to answer an incoming call at aninconvenient time with a known gesture that is translated by the systemtranslation profile as a standardized message to the caller (“Notavailable now, will call back later”, “At work”, etc.).

The aforementioned specific use with a smartphone can easily be expandedto smart tvs, smart glasses, etc.

It is supposed that the present invention is not restricted to any formof realization described previously and that some modifications can beadded to the presented example of fabrication without reappraisal of theappended claims. For example, the present invention has been describedreferring to gaming and some other applications, but it is clear thatthe invention can be applied to more specific applications such asdefense or reconnaissance, surgery, modelling, architecture, and others.

The invention claimed is:
 1. A method for agnostic interaction with ahost system running a type of application, comprising the followingsteps: a. detecting one or more hand manipulations of a user via one ormore haptic devices, and providing one or more motion output signals,associated to the hand manipulations, to a processing unit on the one ormore haptic devices; characterized in that the method comprises thefollowing steps: b. providing a system conversion library to said hostsystem, said system conversion library comprising a plurality of systemtranslation profiles, each system translation profile distinctive to aspecific type of host system, and each system translation profilesuitable for mapping a conversion of motion output signals from the oneor more haptic devices to one or more associated control signalsrecognizable for said specific type of host system, wherein said systemconversion library comprises system translation profiles distinctive toat least two or more of the following types of host system: personalcomputer, two or more specific home video game consoles, smartphone,smart glasses, smart tvs; c. automatically determining the type of thehost system by the processing unit, and wherein in case of inability todetermine the type of the host system, allowing the user to eithermanually determine the type of the host system and/or prompting the userto create a new system translation profile for the undetermined hostsystem, wherein said new system translation profile is stored on theprocessing unit of the haptic device and/or on the host system; d.processing the provided motion output signals in the processing unit onthe one or more haptic devices; e. transmitting the processed motionoutput signals to the host system, via a wireless communicationstandard; f. translating said motion output signals by the host systeminto one or more associated control signals according to the systemtranslation profile distinctive to the host system running theapplication; and g. processing said associated control signals in theapplication and modifying features therein according to at least saidcontrol signals, by the host system.
 2. A method for agnosticinteraction with a host system running a type of application, comprisingthe following steps: a. detecting one or more hand manipulations of auser via one or more haptic devices, and providing one or more motionoutput signals, associated to the hand manipulations, to a processingunit on the one or more haptic devices; characterized in that the methodcomprises the following steps: b. providing a system conversion libraryto said processing unit, said system conversion library comprising aplurality of system translation profiles, each system translationprofile distinctive to a specific type of host system, and each systemtranslation profile suitable for mapping a conversion of motion outputsignals from the one or more haptic devices to one or more associatedcontrol signals recognizable for said specific type of host system,wherein said system conversion library comprises system translationprofiles distinctive to at least two or more of the following types ofhost system: personal computer, two or more specific home video gameconsoles, smartphone, smart tvs, smart glasses; c. automaticallydetermining the type of the host system by the processing unit, andwherein in case of inability to determine the type of the host system,allowing the user to either manually determine the type of the hostsystem and/or prompting the user to create a new system translationprofile for the undetermined host system, wherein said new systemtranslation profile is stored on the processing unit of the hapticdevice and/or on the host system; d. processing the provided motionoutput signals in the processing unit on the one or more haptic devices;e. translating said motion output signals by the processing unit intoone or more associated control signals according to the systemtranslation profile distinctive to the host system running theapplication; f. transmitting the one or more associated control signalsto the host system, via a wireless communication standard; and g.processing said associated control signals in the application andmodifying features therein according to at least said control signals,by the host system.
 3. The method according to claim 1, furthercomprising a step of: a. providing an application conversion library tosaid host system, said application conversion library comprising aplurality of application translation profiles, each applicationtranslation profile distinctive to a specific type of application, andeach application translation profile being adapted for mapping aconversion of received motion output signals from the one or more hapticdevices to one or more associated control signals suitable for saidspecific type of application, wherein said application conversionlibrary comprises application translation profiles distinctive to atleast two or more of the following types of application: shooterapplication, flight simulator application, driving or racingapplication, remote piloting application, smart device interaction;wherein the step of translating said motion output signals into one ormore associated control signals according to the system translationprofile distinctive to the host system running the application, isexecuted furthermore according to the application translation profiledistinctive to the type of the application being run on the host system.4. The method according to claim 2, further comprising a step of: a.providing an application conversion library to said processing unit,said application conversion library comprising a plurality ofapplication translation profiles, each application translation profiledistinctive to a specific type of application, and each applicationtranslation profile being configured for mapping a conversion ofreceived motion output signals from the one or more haptic devices toone or more associated control signals suitable for said specific typeof application, wherein said application conversion library comprisesapplication translation profiles distinctive to at least two or more ofthe following types of application: shooter application, flightsimulator application, driving or racing application, remote pilotingapplication, smart device interaction; wherein the step of translatingsaid motion output signals into one or more associated control signalsaccording to the system translation profile distinctive to the hostsystem running the application, is executed furthermore according to theapplication translation profile distinctive to the type of theapplication being run on the host system.
 5. The method according toclaim 3, comprising a step of the user manually determining theapplication translation profile, from the application conversionlibrary, to be used for translating the motion output signals,furthermore allowing the user to create a new application translationprofile for the type of application running on the host system, whereinsaid new application translation profile is stored on the processingunit of the haptic device and/or on the host system.
 6. The methodaccording to claim 1, wherein the haptic device is a haptic glovesuitable for being worn on a hand of the user, comprising sensors atleast adapted for detecting the following hand manipulations, and theintensity and/or speed and/or acceleration of said following handmanipulations: bending one or more fingers of the hand, relativelydisplacing and/or rotating one or more fingers of the hand with respectto one or more other fingers of the hand, pressure being exerted by aportion of one or more fingers on a surface, pressure being exerted by aportion of the hand on a surface, displacement of the hand with respectto the host system, rotation of the hand with respect to the hostsystem; and wherein said following hand manipulations, and saidintensity and/or speed, is provided to the processing unit as associatedmotion output signals.
 7. The method according to claim 1, wherein theprocessing unit is or is comprised in a smartphone or electronic device,suitable for operable insertion in a virtual reality headset, saidsmartphone or electronic device comprising one or more sensors fordetermining spatial displacement of the user: gyroscope, dead reckoningsensor, magnetometer and/or accelerometer; said method comprising thesteps of: processing displacement signals from said one or more sensorsfor determining spatial displacement; providing additional and/oralternative signals to the host system based on said displacementsignals to the host system; and furthermore modifying features of theapplication, by modifying a virtual reality or an augmented reality bythe host system according to at least additional and/or alternativedisplacement signals.
 8. The method according to claim 6, wherein thehaptic glove is adapted for determining the intensity of said followinghand manipulations, comprising a step of calibrating at least the motionoutput signals associated to the intensity of the bending of one or morefingers of the hand and to the intensity of pressure being exerted by aportion of one or more fingers of the hand.
 9. The method according toclaim 1, wherein said one or more haptic devices comprise a plurality ofreversibly inflatable air chambers on the haptic device for exertingpressure and/or force on a body part of the user, and a plurality ofnano-compressors for inflating and deflating said air chambers, saidmethod comprising a step of transmitting one or more feedback signals tothe processing unit of the one or more haptic devices based on saidmodified features, wherein said feedback signals are actuated byinflation and/or deflation of one or more of the plurality of the airchambers by one or more of the nano-compressors.
 10. The methodaccording to claim 1, wherein the associated control signals are adaptedto replicate expected input signals for the host system.
 11. The methodaccording to claim 1, wherein the step of processing the provided motionoutput signals in the processing unit on the one or more haptic devices,comprises applying a Kalman filtering algorithm on the motion outputsignals.
 12. The method according to claim 3, wherein the systemconversion library and the application conversion library are combinedin a single conversion library, said conversion library comprising aplurality of translation profiles, each translation profile distinctiveto a specific type of application and a specific type of host system andeach translation profile being adapted for mapping a conversion ofreceived motion output signals from the one or more haptic devices toone or more associated control signals recognizable for the specifictype of host system of the translation profile and suitable for thespecific type of application of the translation profile; and wherein thestep of translating said motion output signals into one or moreassociated control signals is executed according to the translationprofile distinctive to the type of host system running the applicationand the type of application being run.
 13. An electronic system forinteracting with an electronic host system running a type ofapplication, and for modifying the application according to receivedcontrol signals, said electronic system for interacting comprising: atleast one at least partly flexible haptic devices suitable to be worn ona user's hand, comprising: i. a plurality of sensors configured fordetecting hand manipulations of the user's hand and transmittingassociated motion output signals, at least comprising: bending of one ormore fingers and pressure being exerted by one or more surface on asurface; ii. a processing unit electronically connected to the pluralityof sensors, said processing unit being adapted for receiving theassociated motion output signals from one or more of the plurality ofsensors, configured for processing the associated motion output signals,said processing unit being configured to recognize the type of hostsystem, and adapted for transmitting the processed motion output signalsto the host system via a wireless communication standard, the processingunit configured for automatically determining the type of the hostsystem by the processing unit, and wherein in case of inability todetermine the type of the host system, allowing the user to eithermanually determine the type of the host system and/or prompting the userto create a new system translation profile for the undetermined hostsystem, wherein said new system translation profile is stored on theprocessing unit of the haptic device and/or on the host system; whereinsaid processing unit is provided with a system conversion library, saidsystem conversion library comprising a plurality of system translationprofiles, each system translation profile distinctive to a specific typeof host system, and each system translation profile suitable for mappinga conversion of motion output signals from the one or more hapticdevices to one or more associated control signals recognizable for saidspecific type of host system, wherein said system conversion librarycomprises system translation profiles distinctive to at least two ormore of the following types of host system: personal computer, homevideo game console, smartphone, dedicated VR and/or AR hardware, smarttvs, smart glasses, robots, drones.
 14. An electronic system forrunning, managing and interacting with a type of application,comprising: a. an electronic host system running the application, andfor modifying the application according to received control signals; b.at least one at least partly flexible haptic devices suitable to be wornon a user's hand, comprising: i. a plurality of sensors configured fordetecting hand manipulations of the user's hand and transmittingassociated motion output signals, at least comprising: bending of one ormore fingers and pressure being exerted by one or more surface on asurface; ii. a processing unit electronically connected to the pluralityof sensors, said processing unit being adapted for receiving theassociated motion output signals from one or more of the plurality ofsensors, configured for processing the associated motion output signals,and adapted for transmitting the processed motion output signals to thehost system via a wireless communication standard, the processing unitconfigured for automatically determining the type of the host system bythe processing unit, and wherein in case of inability to determine thetype of the host system, allowing the user to either manually determinethe type of the host system and/or prompting the user to create a newsystem translation profile for the undetermined host system, whereinsaid new system translation profile is stored on the processing unit ofthe haptic device and/or on the host system; wherein said host system isprovided with a system conversion library, said system conversionlibrary comprising a plurality of system translation profiles, eachsystem translation profile distinctive to a specific type of hostsystem, and each system translation profile suitable for mapping aconversion of motion output signals from the one or more haptic devicesto one or more associated control signals recognizable for said specifictype of host system, wherein said system conversion library comprisessystem translation profiles distinctive to at least two or more of thefollowing types of host system: personal computer, home video gameconsole, smartphone, dedicated VR and/or AR hardware, smart tvs, smartglasses, robots, drones.